<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="description" content="coinfaces">

        <title>numpy数据结构</title>

    <link rel="stylesheet" href="../../theme/css/style.css">
    <link rel="stylesheet" href="../../theme/css/pygments.css">



</head>

<body>
    <div class="content">
<h1>numpy数据结构</h1>    <p>
        under
            <a href="../../tags/numpy/">numpy</a>
    </p>
    <p>
        in <a href="../../categories/tech/">tech</a>
    </p>
    <p>Published: 2017-02-03</p>


    <p>numpy是一个Python高效数值运算库，高效的代价是其数值计算核心要用C和Fortran实现，而在使用numpy时，可能有必要了解更底层的知识，特别是涉及I/O操作的应用。</p>
<p>多维数组（ndarray）是numpy的基础类型，是对多个相同类型的数据的封装，ndarray对这些数据的描述分为两部分：类型和维度，类型描述了每个单元数据表示什么，维度描述这些单元数据之间的相互关系。</p>
<div class="section" id="dtype">
<h2>dtype</h2>
<p>numpy里面的数据类型（data type）术语是dtype，它是对多维数组中单元数据的抽象，在dtype之上，一个整数数组和一个浮点数数组的维度变换和broadcast操作没有任何区别，涉及单元数据的运算时，才需要知道每个单元数据的确切含义。</p>
<p>dtype的任务就是解释单元数据表示的含义，它可以是一个数、一个数值数组、一个字符串或者这些成员的组合，甚至可以包含其它dtype对象，举个例子：</p>
<pre class="literal-block">
 t1                        t1     t1     t1
  |                         |      |      |
  V                         V      V      V
[ int | float array | t2 ][ ... ][ ... ][ ... ] ...
                      |
                      V
                    [ string | complex ]
</pre>
<p>其中t1是包含3个成员，第一个成员是一个整数，第二个成员是一个浮点数组，第三个成员是另一个由字符串和复数组成的数据类型t2。</p>
</div>
<div class="section" id="id1">
<h2>基础类型</h2>
<p>从上面的例子可以看出，dtype用到了整数、浮点数、字符串等基本元素来描述数据类型，这些基本元素来自于numpy定义的一套类型系统，以<tt class="docutils literal">numpy.generic</tt>为根，形成一个树形结构，比如numpy 1.8.2定义了以下类型（省略<tt class="docutils literal">numpy.</tt>前缀）：</p>
<ul class="simple">
<li>generic<ul>
<li>bool8</li>
<li>datetime64</li>
<li>flexible<ul>
<li>character<ul>
<li>string_(str, character)</li>
<li>unicode_(unicode, character)</li>
</ul>
</li>
<li>void<ul>
<li>core.records.record</li>
</ul>
</li>
</ul>
</li>
<li>number<ul>
<li>inexact<ul>
<li>complexfloating<ul>
<li>complex64</li>
<li>complex128(complexfloating, complex)</li>
</ul>
</li>
<li>floating<ul>
<li>float16</li>
<li>float32</li>
<li>float64(floating, float)</li>
</ul>
</li>
</ul>
</li>
<li>integer<ul>
<li>signedinteger<ul>
<li>int8</li>
<li>int16</li>
<li>int32</li>
<li>int64(signedinteger, int)</li>
<li>timedelta64</li>
</ul>
</li>
<li>unsignedinteger<ul>
<li>uint8</li>
<li>uint16</li>
<li>uint32</li>
<li>uint64</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li>object_</li>
</ul>
</li>
</ul>
<p>可以看出，numpy支持布尔、字符和数这些常用类型，其中数分为整数、浮点数和复数，并根据精度进一步细分，其后缀数字即该类型的总位数。例如<tt class="docutils literal">float32</tt>，表示该类型的浮点数占32位，即4字节。</p>
<p>继承了Python内置类型的类型右侧括号里标出了详细的继承关系，例如<tt class="docutils literal">string_</tt>类型，从树形结构中的位置来看，当然继承自<tt class="docutils literal">character</tt>，但它同时也继承自Python内置类型<tt class="docutils literal">str</tt>。</p>
<p>名字中带<tt class="docutils literal">_</tt>后缀的类型，经测试有以下关系成立：</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">numpy</span> <span class="kn">import</span> <span class="p">(</span><span class="n">bool_</span><span class="p">,</span>
                   <span class="n">complex_</span><span class="p">,</span>
                   <span class="n">dtype</span><span class="p">,</span>
                   <span class="n">float_</span><span class="p">,</span>
                   <span class="n">int_</span><span class="p">,</span>
                   <span class="n">object_</span><span class="p">,</span>
                   <span class="n">str_</span><span class="p">,</span>
                   <span class="n">string_</span><span class="p">,</span>
                   <span class="n">unicode_</span><span class="p">)</span>
<span class="n">dtype</span><span class="p">(</span><span class="n">bool_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="nb">bool</span><span class="p">)</span>
<span class="n">dtype</span><span class="p">(</span><span class="n">string_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="n">str_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span>
<span class="n">dtype</span><span class="p">(</span><span class="n">unicode_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="n">unicode</span><span class="p">)</span>
<span class="n">dtype</span><span class="p">(</span><span class="n">complex_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="nb">complex</span><span class="p">)</span>
<span class="n">dtype</span><span class="p">(</span><span class="n">float_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="nb">float</span><span class="p">)</span>
<span class="n">dtype</span><span class="p">(</span><span class="n">int_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="nb">int</span><span class="p">)</span>
<span class="n">dtype</span><span class="p">(</span><span class="n">object_</span><span class="p">)</span> <span class="o">==</span> <span class="n">dtype</span><span class="p">(</span><span class="nb">object</span><span class="p">)</span>
</pre></div>
<p>除了<tt class="docutils literal">bool_</tt>和<tt class="docutils literal">object_</tt>，这些类型都继承自相应的Python内置类型。</p>
<p>令人迷惑的是，numpy这个名空间里，也定义了<tt class="docutils literal">bool</tt>，<tt class="docutils literal">complex</tt>、<tt class="docutils literal">float</tt>、<tt class="docutils literal">int</tt>、<tt class="docutils literal">object</tt>，<tt class="docutils literal">str</tt>和<tt class="docutils literal">unicode</tt>等名字，但它们只是Python内置变量的别名。</p>
</div>
<div class="section" id="id2">
<h2>dtype和基础类型的区别</h2>
<p>dtype对象和基础类型都可以用来指定数组的数据类型，比如以下两个整数数组的定义都是合法的：</p>
<div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="n">np</span><span class="o">.</span><span class="n">int32</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="n">dtype</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">int32</span><span class="p">))</span>
</pre></div>
<p>但两者在概念上完全不同，基础类型是Python类，例如上面定义的数组<tt class="docutils literal">x</tt>的每个元素都是<tt class="docutils literal">np.int32</tt>的实例。<tt class="docutils literal">np.int32</tt>本身也可以实例化：<tt class="docutils literal">np.int32(5)</tt>。</p>
<p>dtype对象不是类型，而是<tt class="docutils literal">dtype</tt>类的实例，其作用是描述数组中每个元素的内存结构。通过<tt class="docutils literal">dtype</tt>处理新类型的数组时，只要定义新的dtype对象，而不需要创建新类。</p>
<p>如果指定的类型不是dtype对象，可以视作用dtype构造函数将其转化成一个dtype对象，dtype构造函数接受的描述方式非常多样化，为了减轻记忆负担并维持风格的统一，这里提供一种实用的dtype描述方法，举例如下：</p>
<ul>
<li><p class="first">基础数值类型：<tt class="docutils literal">np.int32</tt>、<tt class="docutils literal">np.float64</tt></p>
</li>
<li><p class="first">可变长度类型：<tt class="docutils literal">(np.string_, 8)</tt></p>
</li>
<li><p class="first">数值数组：<tt class="docutils literal">(np.int8, (5, 4))</tt></p>
</li>
<li><p class="first">多个不同类型/形状的组合：</p>
<pre class="literal-block">
[('a', np.int16),
 ('b', np.float32, 2),
 ('c', np.string_, 5)]
</pre>
</li>
</ul>
<p>避免使用像<tt class="docutils literal">&quot;int32&quot;</tt>这样的字符串，更要避免使用<tt class="docutils literal">&quot;i4&quot;</tt>这样的简写，因为字符串容易打错，不能自动补齐。</p>
<p>在指定比较复杂的结构时，使用<tt class="docutils literal">list</tt>，尽量不要使用<tt class="docutils literal">dict</tt>类型，因为后者无法自动确定结构各成员的先后顺序，要手动输入每个成员的内存地址偏移量，容易出错并且难以向已定义的结构体中插入新成员，也要避免使用<tt class="docutils literal">(S4,f8)</tt>这样的字符串，原因同上。</p>
</div>
<div class="section" id="id3">
<h2>结构体</h2>
<p>可以看到，numpy可以非常方便地定义结构体，并且结构体的嵌套也很容易实现，比如以下C结构体：</p>
<div class="highlight"><pre><span></span><span class="k">typedef</span> <span class="k">struct</span> <span class="p">{</span>
    <span class="kt">int</span> <span class="n">difficulty</span><span class="p">;</span>
    <span class="kt">int</span> <span class="n">performance</span><span class="p">;</span>
<span class="p">}</span> <span class="n">_score</span><span class="p">;</span>

<span class="k">typedef</span> <span class="k">struct</span> <span class="p">{</span>
    <span class="kt">char</span> <span class="n">name</span><span class="p">[</span><span class="mi">256</span><span class="p">];</span>
    <span class="kt">char</span> <span class="n">sex</span><span class="p">[</span><span class="mi">10</span><span class="p">];</span>
    <span class="n">_score</span> <span class="n">score</span><span class="p">[</span><span class="mi">20</span><span class="p">];</span>
<span class="p">}</span> <span class="n">_player</span><span class="p">;</span>
</pre></div>
<p>可以被直观地表示为：</p>
<div class="highlight"><pre><span></span><span class="n">score_type</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dtype</span><span class="p">([(</span><span class="s2">&quot;difficulty&quot;</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">int32</span><span class="p">),</span>
                       <span class="p">(</span><span class="s2">&quot;performance&quot;</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">int32</span><span class="p">)])</span>
<span class="n">player_type</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dtype</span><span class="p">([(</span><span class="s2">&quot;name&quot;</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">string_</span><span class="p">,</span> <span class="mi">256</span><span class="p">),</span>
                        <span class="p">(</span><span class="s2">&quot;sex&quot;</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">string_</span><span class="p">,</span> <span class="mi">10</span><span class="p">),</span>
                        <span class="p">(</span><span class="s2">&quot;score&quot;</span><span class="p">,</span> <span class="n">score_type</span><span class="p">,</span> <span class="mi">20</span><span class="p">)])</span>
</pre></div>
<p>该结构体的每个成员都可以被单独取出：</p>
<div class="highlight"><pre><span></span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">ndarray</span><span class="p">((</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="n">dtype</span><span class="o">=</span><span class="n">player_type</span><span class="p">)</span>
<span class="n">x</span><span class="p">[</span><span class="s1">&#39;name&#39;</span><span class="p">]</span>  <span class="c1"># 2x2 array</span>
<span class="n">x</span><span class="p">[</span><span class="s1">&#39;score&#39;</span><span class="p">][</span><span class="s1">&#39;difficulty&#39;</span><span class="p">]</span>  <span class="c1"># 2x2x20 array</span>
</pre></div>
<p>结构体数组可以看成一个数组，其每个成员都具有复杂的结构，或者看成多个数组嵌套组合在一起，其每个成员都具有相同的维度。从这个意义上说，结构本身也可以看成一个独立的维度（在numpy里确实如此，并且可以用整数来索引结构体成员），只是这个维度可以用成员的名字来索引，在用名字索引时，该维度出现的位置并不重要，<tt class="docutils literal"><span class="pre">x['name'][1]</span></tt>和<tt class="docutils literal"><span class="pre">x[1]['name']</span></tt>是一样的。</p>
<p>numpy还提供了<tt class="docutils literal">recarray</tt>类型，可以通过属性来获取结构体成员，但这些属性是动态的，还无法做到自动补齐，在我看来，只是少打几个符号而已。</p>
</div>

    </div>
<nav class="nav sidebar">
    <div class="siteinfo">
        <div class="sitename"><a href="../..">coinfaces</a></div>
        <div class="tagline">Another Side of Life</div>
    </div>

        <a href="../../categories/">categories</a>
        <a href="../..">&larr; Home</a>
</nav><footer class="footer">
    <span>
        &copy; jpli &middot;
        under <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC-BY-SA 4.0</a>
    </span>
    |
    <span>
        Powered by:
        <a href="http://blog.getpelican.com/">Pelican</a>
    </span>
</footer></body>
</html>